Floating Point Arithmetic
Digital electronic devices, such as digital computers, calculators, and other devices, perform arithmetic calculations on values in integer, or “fixed point,” format, in fractional, or “floating point” format, or both. IEEE Standard 754, (hereinafter “IEEE Std. 754” or “the Standard”) published in 1985 by the Institute of Electrical and Electronic Engineers, and adopted by the American National Standards Institute (ANSI), defines several standard formats for expressing values in floating point format, and a number of aspects regarding the behavior of computations in connection therewith. In accordance with IEEE Std. 754, a representation in floating point format comprises a plurality of binary digits, or “bits,” having the structuresemsb . . . elsbfmsb . . . flsbwhere bit “s” is a sign bit indicating whether the entire value is positive or negative, bits “emsb . . . elsb” comprise an exponent field representing the exponent “e” in unsigned binary biased format, and bits “fmsb . . . flsb” comprise a fraction field that represents the fractional portion “f” in unsigned binary format (“msb” represents “most significant bit” and “lsb” represents “least significant bit”). The Standard defines two general formats, namely, a “single” format which comprises thirty-two bits, and a “double” format which comprises sixty-four bits. In the single format, there is one sign bit “s,” eight bits “e7 . . . e0” comprising the exponent field and twenty-three bits “f22 . . . f0” comprising the fraction field. In the double format, there is one sign bit “s,” eleven bits “e10 . . . e0” comprising the exponent field and fifty-two bits “f51 . . . f0” comprising the fraction field.
As indicated above, the exponent field of the floating point representation “emsb . . . elsb” represents the exponent “E” in biased format. The biased format provides a mechanism by which the sign of the exponent is implicitly indicated. In particular, the bits “emsb . . . elsb” represent a binary encoded value “e” such that “e=E+bias.” This allows the exponent E to extend from −126 to +127, in the eight-bit “single” format, and from −1022 to +1023 in the eleven-bit “double” format, and provides for relatively easy manipulation of the exponents in multiplication and division operations, in which the exponents are added and subtracted, respectively.
IEEE Std. 754 provides for several different formats with both the single and double formats, which are generally based on the bit patterns of the bits emsb . . . elsb comprising the exponent field and the bits fmsb . . . flsb comprising the fraction field. As shown in prior art FIG. 1, if a number is represented such that all of the bits emsb . . . elsb of the exponent field are binary ones (that is, if the bits represent a binary-encoded value of “255” in the single format or “2047” in the double format) and all of the bits fmsb . . . flsb of the fraction field are binary zeros, then the value of the number is positive infinity 110 or negative infinity 120, depending on the value of the sign bit “s.” In particular, the value “v” is v=(−1)s∞ where “∞” represents the value “infinity.” On the other hand, if the bit pattern is formatted such that all of the bits emsb . . . elsb of the exponent field are binary ones and the bits fmsb . . . flsb of the fraction field are not all zeros, then the value that is represented is deemed “not a number,” abbreviated in the Standard by “NaN” 130.
If a number has an exponent field in which the bits emsb . . . elsb are neither all binary ones nor all binary zeros (that is, if the bits represent a binary-encoded value between 1 and 254 in the single format or between 1 and 2046 in the double format), the number is said to be in a “normalized” format 160. For a number in the normalized format, the value represented by the number is v=(−1)s2e-bias(1.|fmsb . . . flsb), where “|” represents a concatenation operation. Effectively, in the normalized format, there is an implicit most significant digit having the value “one,” so that the twenty-three digits in the fraction field of the single format, or the fifty-two digits in the fraction field of the double format, will effectively represent a value having twenty-four digits or fifty-three digits of precision, respectively, where the value is less than two, but not less than one.
On the other hand, if a number has an exponent field in which the bits emsb . . . elsb are all binary zeros, representing the binary-encoded value of “zero,” and a fraction field in which the bits fmsb . . . flsb are not all zero, the number is said to be in a “denormalized” format 170. For a number in the denormalized format, the value represented by the number is v=(−1)s2e-bias+1(0.|fmsb . . . flsb). It will be appreciated that the range of values of that can be expressed in the denormalized format is disjoint from the range of values of numbers that can be expressed in the normalized format, for both the single and double formats. Finally, if a number has an exponent field in which the bits emsb . . . elsb are all binary zeros, representing the binary-encoded value of “zero,” and a fraction field in which the bits fmsb . . . flsb are all zero, the number has the value “zero.” It will be appreciated that the value “zero” may be positive zero 140 or negative zero 150, depending on the value of the sign bit.
Generally, floating point units to perform computations whose results conform to IEEE Std. 754 are designed to generate a result in response to a floating point instruction in three steps:
(a) First, an approximation calculation step in which an approximation to the absolutely accurate mathematical result (assuming that the input operands represent the specific mathematical values as described by IEEE Std. 754) is calculated that is sufficiently precise as to allow this accurate mathematical result to be summarized by a sign bit, an exponent (typically represented using more bits than are used for an exponent in the standard floating-point format), and some number “N” of bits of the presumed result fraction, plus a guard bit and a sticky bit. The value of the exponent will be such that the value of the fraction generated in step (a) consists of a 1 before the binary point and a fraction after the binary point. The bits are calculated so as to obtain the same result as the following conceptual procedure (which is impossible under some circumstances to carry out in practice): calculate the mathematical result to an infinite number of bits of precision in binary scientific notation, and in such a way that there is no bit position in the significand such that all bits of lesser significance are 1-bits (this restriction avoids the ambiguity between, for example, 1.100000 . . . and 1.011111 . . . as representations of the value “one-and-one-half”); then let the N most significant bits of the infinite significand be used as the intermediate result significand, let the next bit of the infinite significand be the guard bit, and let the sticky bit be 0 if and only if ALL remaining bits of the infinite significand are 0-bits (in other words, the sticky bit is the logical OR of all remaining bits of the infinite fraction after the guard bit).
(b) Second, a rounding step, in which the guard bit, the sticky bit, perhaps the sign bit, and perhaps some of the bits of the presumed significand generated in step (a) are used to decide whether to alter the result of step (a). For the rounding modes defined by IEEE Std. 754, this is a decision as to whether to increase the magnitude of the number represented by the presumed exponent and fraction generated in step (a). Increasing the magnitude of the number is done by adding 1 to the significand in its least significant bit position, as if the significand were a binary integer. It will be appreciated that, if the significand is all 1-bits, then magnitude of the number is “increased” by changing it to a high-order 1-bit followed by all 0-bits and adding 1 to the exponent. It will be further appreciated that,
(i) if the result is a positive number, and                (a) if the decision is made to increase, effectively the decision has been made to increase the value of the result, thereby rounding the result up, towards positive infinity, but        (b) if the decision is made not to increase, effectively the decision has been made to decrease the value of the result, thereby rounding the result down, towards negative infinity; and        
(ii) if the result is a negative number, and                (a) if the decision is made to increase, effectively the decision has been made to decrease the value of the result, thereby rounding the result down, towards negative infinity, but        (b) if the decision is made not to increase, effectively the decision has been made to increase the value of the result, thereby rounding the result up, towards positive infinity.        
(c) Finally, a packaging step, in which the result is packaged into a standard floating-point format. This may involve substituting a special representation, such as the representation format defined for infinity or NaN if an exceptional situation (such as overflow, underflow, or an invalid operation) was detected. Alternatively, this may involve removing the leading 1-bit (if any) of the fraction, because such leading 1-bits are implicit in the standard format. As another alternative, this may involve shifting the fraction in order to construct a denormalized number. As a specific example, we assume that this is the step that forces the result to be a NaN if any input operand is a NaN. In this step, the decision is also made as to whether the result should be an infinity. It will be appreciated that, if the result is to be a NaN or infinity, any result from step (b) will be discarded and instead the appropriate representation in the appropriate format will be provided as the result.
In addition in the packaging step, floating-point status information is generated, which is stored in a floating point status register. The floating point status information generated for a particular floating point operation includes indications, for example, as to whether:                (i) a particular operand is invalid for the operation to be performed (“invalid operation”);        (ii) if the operation to be performed is division, the divisor is zero (“division-by-zero”);        (iii) an overflow occurred during the operation (“overflow”);        (iv) an underflow occurred during the operation (“underflow”); and        (v) the rounded result of the operation is not exact (“inexact”).        
These conditions are typically represented by flags that are stored in the floating point status register, separate from the floating point operand. The floating point status information can be used to dynamically control the operations performed in response to certain instructions, such as conditional branch, conditional move, and conditional trap instructions that may be in the instruction stream subsequent to the floating point instruction. Also, the floating point status information may enable processing of a trap sequence, which will interrupt the normal flow of program execution. In addition, the floating point status information may be used to affect certain ones of the functional unit control signals that control the rounding mode. IEEE Std. 754 also provides for accumulating floating point status information from, for example, results generated for a series or plurality of floating point operations.
IEEE Std. 754 has brought relative harmony and stability to floating-point computation and architectural design of floating-point units. Moreover, its design was based on some important principles and rests on sensible mathematical semantics that ease the job of programmers and numerical analysts. It also provides some support for the implementation of interval arithmetic, which may prove to be preferable to simple scalar arithmetic for many tasks. Nevertheless, IEEE Std. 754 has some serious drawbacks, including:
(i) Modes, which include the rounding mode and may also include a traps enabled/disabled mode, flags representing the floating point status information that is stored in the floating point status register, and traps that are required to implement IEEE Std. 754, all introduce implicit serialization between floating-point instructions, and between floating point instructions and the instructions that read and write the flags and modes. Implicit serialization occurs when programmers and designers try to avoid the problems caused if every floating point instructions uses, and can change, the same floating point status register. This can create problems if, for example, two instructions are executing in parallel in a microprocessor architectures featuring several CPUs running at once and both instructions cause an update of the floating point status register. In such a case, the contents of the status register would likely be incorrect with respect to at least one of the instructions, because the other parallel instruction will have written over the original contents. Similar problems can occur in scalar processor architectures, in which several instructions are issued and processed at once. To solve this problem, programmers and designers serialize floating point instructions that can affect the floating point status register, making sure they execute in a serial fashion, one instruction completing before another begins. Rounding modes can introduce implicit serialization because they are typically indicated as global state, although in some microprocessor architectures, the rounding mode is encoded as part of the instruction operation code, which will alleviate this problem to that extent. This implicit serialization makes the Standard difficult to implement coherently in today's superscalar and parallel microprocessor architectures without loss of performance.
(ii) The implicit side effects of a procedure that can change the flags or modes can make it very difficult for compilers to perform optimizations on floating-point code; to be safe, compilers for most languages must assume that every procedure call is an optimization barrier.
(iii) Global flags, such as those that signal certain modes, make it more difficult to do instruction scheduling where the best performance is provided by interleaving instructions of unrelated computations. Instructions from regions of code governed by different flag settings or different flag detection requirements cannot easily be interleaved when they must share a single set of global flag bits in a global floating point status register.
(iv) Traps have been difficult to integrate efficiently into architectures and programming language designs for fine-grained control of algorithmic behavior.
U.S. patent application Ser. No. 10/035,584, filed on even date herewith in the name of Guy L. Steele Jr. and entitled “Floating Point Unit In Which Floating Point Status Information Is Encoded In Floating Point Representations,” describes a floating point unit in which floating point status information is encoded in the representations of the results generated thereby. By encoding the floating point status information relating to a floating point operation in the result that is generated for the operation, the implicit serialization required by maintaining the floating point status information separate and apart therefrom can be obviated. The floating point unit includes a plurality of functional units, including an adder unit, a multiplier unit, a divider unit, a square root unit, a maximum/minimum unit, a comparator unit and a tester unit, all of which operate under control of functional unit control signals provided by a control unit. It may also include features consistent with the principles of the present invention to provide better support for interval arithmetic.
Interval Arithmetic
Interval arithmetic can be used when processing floating point operands to compute an interval result. In general, an interval is the set of all real numbers between and including the lower and upper bound of the interval. Interval arithmetic is used to evaluate arithmetic expressions over sets of numbers contained in intervals. An interval arithmetic result is a new interval that contains the set of all possible resulting values.
In interval arithmetic computations, each computed value is represented as a pair of numbers [a, b] (where a<=b) that indicates a lower bound and an upper bound of an interval on the real number line. One can regard an interval [a, b] as a set:{p|a<=p<=b}
It is also convenient to use “−Infinity” and “+Infinity” as bounds. For example, [3, +Infinity] represents the set of all numbers not less than 3.
Those skilled in the art will recognize that the most general definition of a binary operation f on two intervals x and y in terms of an underlying binary operation (also called f) on real numbers is:[glb S, lub S]where S={f(p,q)|p in x and q in y}
That is, one considers all possible pairs of real arguments where each argument ranges over an input interval and computes f on all these possible pairs; the result is a set S of real results. The set S may fail to be contiguous, so to produce an interval result it is necessary to “fill it in.” The greatest lower bound (glb) of S and the least upper bound (lub) of S may be used as endpoints of the result interval.
For particular “well-behaved” binary operations, it is not necessary to consider an infinite set of computations to compute the endpoints of the result interval. In particular, intervals can be added, subtracted, multiplied, and divided using the formulae:
                                          [                          a              ,              b                        ]                    +                      [                          c              ,              d                        ]                          =                ⁢                  [                                    a              +              c                        ,                          b              +              d                                ]                                                              [                          a              ,              b                        ]                    -                      [                          c              ,              d                        ]                          =                ⁢                  [                                    a              -              d                        ,                          b              -              c                                ]                                                              [                          a              ,              b                        ]                    *                      [                          c              ,              d                        ]                          =                ⁢                  [                                    min              ⁢                              (                                                      a                    *                    c                                    ,                                      a                    *                    d                                    ,                                      b                    *                    c                                    ,                                      b                    *                    d                                                  )                                      ,                                                           ⁢                  max          ⁡                      (                                          a                *                c                            ,                              a                *                d                            ,                              b                *                c                            ,                              b                *                d                                      )                          ]                                                      [                          a              ,              b                        ]                    /                      [                          c              ,              d                        ]                          =                ⁢                  [                                    min              ⁡                              (                                                      a                    /                    c                                    ,                                      a                    /                    d                                    ,                                      b                    /                    c                                    ,                                      b                    /                    d                                                  )                                      ,                                                           ⁢                  max          ⁡                      (                                          a                /                c                            ,                              a                /                d                            ,                              b                /                c                            ,                              b                /                d                                      )                          ]                                         ⁢                                            provided                        ⁢                                                  ⁢            c                    >                      0            ⁢                                                  ⁢                          or                        ⁢                                                  ⁢            d                    <          0                    
If the numbers are represented in a floating-point representation with finite precision, then addition and multiplication of such numbers produce only approximate results. However, by controlling the rounding of such results, floating-point arithmetic can be correctly used for interval computations. The typical consequence of the floating-point approximations is only that result intervals may be slightly larger than mathematically necessary.
Standard IEEE 754 supports an implementation of interval arithmetic by providing rounding modes such as “round toward plus infinity” and “round toward minus infinity.”
For purposes of the following discussion, let “+UP” denote floating-point addition using rounding mode “round toward plus infinity.” Let “−UP” denote floating-point subtraction using rounding mode “round toward plus infinity.” Let “*UP” denote floating-point multiplication using rounding mode “round toward plus infinity.” Let “/UP” denote floating-point division using rounding mode “round toward plus infinity.” Let “+DOWN” denote floating-point addition using rounding mode “round toward minus infinity.” Let “−DOWN” denote floating-point subtraction using rounding mode “round toward minus infinity.” Let “*DOWN” denote floating-point multiplication using rounding mode “round toward minus infinity.” Finally, let “/DOWN” denote floating-point division using rounding mode “round toward minus infinity.”
Under these naming conventions, the interval computation rules using floating-point arithmetic may be written as:
                                          [                          a              ,              b                        ]                    +                      [                          c              ,              d                        ]                          =                ⁢                  [                                    a              +                              DOWN                ⁢                                                                  ⁢                c                                      ,                          b              +                              UP                ⁢                                                                  ⁢                d                                              ]                                                              [                          a              ,              b                        ]                    -                      [                          c              ,              d                        ]                          =                ⁢                  [                                    a              -                              DOWN                ⁢                                                                  ⁢                d                                      ,                          b              -                              UP                ⁢                                                                  ⁢                c                                              ]                                                              [                          a              ,              b                        ]                    *                      [                          c              ,              d                        ]                          =                ⁢                  [                      min            (                                          a                *                DOWN                ⁢                                                                  ⁢                c                            ,                              a                *                DOWN                ⁢                                                                  ⁢                d                            ,                              b                *                DOWN                ⁢                                                                  ⁢                c                            ,                                                                                   ⁢                      b            *            DOWN            ⁢                                                  ⁢            d                    )                ,                  max          (                                    a              *              UP              ⁢                                                          ⁢              c                        ,                          a              *              UP              ⁢                                                          ⁢              d                        ,                          b              *              UP              ⁢                                                          ⁢              c                        ,                                                                       ⁢                      b            *            UP            ⁢                                                  ⁢            d                    )                ]                                                      [                          a              ,              b                        ]                    /                      [                          c              ,              d                        ]                          =                ⁢                              if                    ⁢                                          ⁢                      (                          c              >                              0                ⁢                                                                  ⁢                                  or                                ⁢                                                                  ⁢                d                            <              0                        )                    ⁢                                          ⁢                      then                                                           ⁢                  [                      min            (                                                            a                  /                  DOWN                                ⁢                                                                  ⁢                c                            ,                                                a                  /                  DOWN                                ⁢                                                                  ⁢                d                            ,                                                b                  /                  DOWN                                ⁢                                                                  ⁢                c                            ,                                                                                   ⁢                                    b              /              DOWN                        ⁢                                                  ⁢            d                    )                ,                  max          (                                                    a                /                UP                            ⁢                                                          ⁢              c                        ,                                          a                /                UP                            ⁢                                                          ⁢              d                        ,                                          b                /                UP                            ⁢                                                          ⁢              c                        ,                                                                       ⁢                                    b              /              UP                        ⁢                                                  ⁢            d                    )                ]                                         ⁢                                            else                        ⁢                                                  [                                          -                Infinity                            ,                              +                Infinity                                      ]                    .                    
Despite this interval arithmetic support, IEEE 754-1985 has some drawbacks for performing interval arithmetic operations. There are certain situations in which IEEE 754 specifies that a floating-point operation must deliver a result in the NaN format, even though the rounding mode is “round toward plus infinity” or “round toward minus infinity” and even though a reasonable result could be returned that would be useful as an interval bound. For example, consider that two intervals:[−Inf, 6] and [+Inf, +Inf].
The sum of these two intervals may be computed as:[−Inf+DOWN+Inf, 6+UP+Inf].                 where “Inf” means “infinity” as specified by IEEE 754. It is clear that a reasonable result would be:[−Inf, +Inf].        
However, IEEE 754 specifies that, under any rounding mode, the result of (−Inf)+(+Inf) is a NaN result. Therefore, using IEEE 754 arithmetic to compute this interval sum produces:[NaN, +Inf],
which is not a properly formed interval. This may undesirably lead to confusing or invalid results for interval arithmetic operations and, potentially, to erroneous calculations.
To prevent malformed intervals such as in the previous example, programmers of IEEE 754 machines typically design special software routines to recognize inputs that create such malformed intervals and use extra code routines to handle them before doing the interval arithmetic operation. Disadvantageously, such special routines waste time and space in software programs.
Thus, there is a need for systems and methods that efficiently support addition, subtraction, multiplication, and division of intervals using floating-point arithmetic similar to that of IEEE 754, but such that the result of every interval operation is a well-formed interval.